Approximate solutions to constrained risk-sensitive Markov decision processes

This paper considers the problem of finding near-optimal Markovian randomized (MR) policies for finite-state-action, infinite-horizon, constrained risk-sensitive Markov decision processes (CRSMDPs). Constraints are in form standard expected discounted cost functions as well over finite and infinite horizons. We first show that aforementioned CRSMDP optimization possesses a solution if it is feasible (that is, there exists policy which satisfies all constraints). Secondly, we provide two methods an approximate ultimately stationary (US) MR policy. The latter achieved through approximating finite-horizon CRSMDPs constructed from original by time-truncating objective constraint functions, suitably perturbing upper bounds. approximation gives US ϵ-optimal problem, while second whose violation constraints bounded above specified tolerance value ϵ. A key step proofs appropriate choice metric makes set infinite-horizon regions three compact, continuous. also discuss applications use inventory control example to illustrate how existing techniques may be used solve problems mentioned above.